Predicting the expected behavior of agents that learn about agents: the CLRI framework

We describe a framework and equations used to model and predict the behavior of multi-agent systems (MASs) with learning agents. A difference equation is used for calculating the progression of an agent's error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agent's learning abilities, such as its change rate, learning rate and retention rate, as well as relevant aspects of the MAS such as the impact that agents have on each other. We validate the framework with experimental results using reinforcement learning agents in a market system, as well as with other experimental results gathered from the AI literature. Finally, we use PAC-theory to show how to calculate bounds on the values of the learning parameters

08461 Abstracts Collection -- Planning in Multiagent Systems

From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 \u27`Planning in Multiagent Systems\u27\u27 was held in Schloss Dagstuhl~--~Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Improving Learning Performance by Applying Economic Knowledge

Digital information economies require information goods producers to learn how to position themselves within a potentially vast product space. Further, the topography of this space is often nonstationary, due to the interactive dynamics of multiple producers changing their position as they try to learn the distribution of consumer preferences and other features of the problem's economic structure. This presents a producer or its agent with a difficult learning problem: how to locate profitable niches in a very large space. In this paper, we present a model of an information goods duopoly and show that, under complete information, producers would prefer not to compete, instead acting as local monopolists and targeting separate niches in the consumer population. However, when producers have no information about the problem they are solving, it can be quite difficult for them to converge on this solution. We show how a modest amount of economic knowledge about the problem can make it much easier, either by reducing the search space, starting in a useful area of the space, or introducing a gradient. These experiments support the hypothesis that a producer using some knowledge of a problem's (economic) structure can outperform a producer that is performing a naive, knowledge-free form of learning.

Model Selection in an Information Economy : Choosing what to Learn

As online markets for the exchange of goods and services become more common, the study of markets composed at least in part of autonomous agents has taken on increasing importance. In contrast to traditional completeinformation economic scenarios, agents that are operating in an electronic marketplace often do so under considerable uncertainty. In order to reduce their uncertainty, these agents must learn about the world around them. When an agent producer is engaged in a learning task in which data collection is costly, such as learning the preferences of a consumer population, it is faced with a classic decision problem: when to explore and when to exploit. If the agent has a limited number of chances to experiment, it must explicitly consider the cost of learning (in terms of foregone profit) against the value of the information acquired. Information goods add an additional dimension to this problem; due to their flexibility, they can be bundled and priced according to a number of different price schedules. An optimizing producer should consider the profit each price schedule can extract, as well as the difficulty of learning of this schedule. In this paper, we demonstrate the tradeoff between complexity and profitability for a number of common price schedules. We begin with a one-shot decision as to which schedule to learn. Schedules with moderate complexity are preferred in the short and medium term, as they are learned quickly, yet extract a significant fraction of the available profit. We then turn to the repeated version of this one-shot decision and show that moderate complexity schedules, in particular two-part tariff, perform well when the producer must adapt to nonstationarity in the consumer population. When a producer can dynamically change schedules as it learns, it can use an explicit decision-theoretic formulation to greedily select the schedule which appears to yield the greatest profit in the next period. By explicitly considering the both the learnability and the profit extracted by different price schedules, a producer can extract more profit as it learns than if it naively chose models that are accurate once learned.Online learning; information economics; model selection; direct search

Commitment-driven distributed joint policy search

Decentralized MDPs provide powerful models of interactions in multi-agent environments, but are often very difficult or even computationally infeasible to solve optimally. Here we develop a hierarchical approach to solving a restricted set of decentralized MDPs. By forming commitments with other agents and modeling these concisely in their local MDPs, agents effectively, efficiently, and distributively formulate co-ordinated local policies. We introduce a novel construction that captures commitments as constraints on local policies and show how Linear Programming can be used to achieve local optimality subject to these constraints. In contrast to other commitment enforcement approaches, we show ours to be more robust in capturing the intended commitment semantics while maximizing local utility. We also describe a commitment-space heuristic search algorithm that can be used to approximate optimal joint policies. A preliminary empirical evaluation suggests that our approach yields faster approximate solutions than the conventional encoding of the problem as a multiagent MDP would allow and, when wrapped in an exhaustive commitment-space search, will find the optimal global solution

A Formal Study of Distributed Meeting Scheduling

Automating routine organizational tasks, such as meeting scheduling, requires a careful balance between the individual (respecting his or her privacy and personal preferences) and the organization (making efficient use of time and other resources). We argue that meeting scheduling is an inherently distributed process, and that negotiating over meetings can be viewed as a distributed search process. Keeping the process tractable requires introducing heuristics to guide distributed schedulers' decisions about what information to exchange and whether or not to propose the same tentative time for several meetings. While we have intuitions about how such heuristics could affect scheduling performance and efficiency, verifying these intuitions requires a more formal model of the meeting schedule problem and process. We present our preliminary work toward this goal, as well as experimental results that validate some of the predictions of our formal model. We also investigate scheduling in overconstrained situations, namely, scheduling of high priority meetings at short notice, which requires cancellation and rescheduling of previously scheduled meetings. Our model provides a springboard into deeper investigations of important issues in distributed artificial intelligence as well, and we outline our ongoing work in this direction.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/42829/1/10726_2004_Article_153020.pd